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ABSTRACT 

Assembly Bill (AB) 1725 mandates that the Board of 
Governors of the California Community Colleges (CCC) shall develop 
and implement a comprehensive Community College educational and 
fiscal accountability system in order to improve student and 
institutional performance. The system is to be implemented over a 
3-year period beginning not later them 1991-92 and is to produce a 
published report of CCC accountabi.Uty. In November 1990, a task 
force was established to provide consultation on the conduct of a 
pilot accountability program, and in January 1991 grants were awarded 
to four ccmmunity College Districts for an 18-month project to 
develop pilot local and state accountability programs and prototype 
accountability reports. Three reports emerged from the pilot 
accountability program: a technical assistance guide for designing 
local pilot accountability programs; a study of state and local 
accountability implementation costs; and a draft plan for a statewide 
accountability program. Aspects of the proposed state accountability 
system would include basic data collection and reporting by the CCC 
Chancellor's Office; annual long-range accountability studies; 
improved data collection and dissemination efforts; and establishment 
of a clearinghouse of exemplary accountability models. The five 
proposed statewide indicators include student access, student 
success, student satisfaction, staff composition, and fiscal 
condition. Appendixes provide tasK force members, a summary of 
components of the technical assistance guide, and a detailed listing 
and description cf proposed accountability indicators for statewide 
reporting. (JMC) 
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CALIFORNIA COMMUNITY COLLEGES 



A Report 



Background 

Assembly Bill 1725 (Chapter 973, Statutes of 1988) mandates that". . . the Board of 
Governors shall develop and implement a comprehensive Community College educa- 
tional and fiscal accounUbility system ..." [Education Code Section 71020.51, in 
order to improve the quality of student and institutional performance. The system is 
to be implemented over a three-year period beginning not later than 1991-92 and is to 
produce a published report of California Community College accountability. 

I 

Analysis 

The Board of Governors adopted a report in July 1990 that discussed some facets of 
the Community College accountability system. This adopUon cleared the way for a 
pilot program. A task force was formed in November. 1990 to provide consuluUon on 
the conduct of the pilot program. (See Appendix A for a list of current task force 
members.) Grants were awarded in January 1991 to four Community College 
DistricU for an eighteen-month effort to develop local and sUte accountability 
programs. 

To date, three reports have been issued including a technical assistance guide for 
designing local accountability programs, a study of state and local accountability 
implementation costs, and a draft plan for a statewide accounUbility program. The 
three reports were written in a coordinated fashion over the first six months of the 
pilot program in consulUUon with the task force. AddiUonal feedback was received 
from the Northern California Community College Research Group (NORCAL), the 
Southern California Community College Institutional Research Association 
(SCCCIRA). and the Community College League of California Research and Plan- 
ning Advisory Committee. Comments were also received at presentations of work m 
progress during meeUngs of various Comir»unity College professional associaUons 
and from interested Colleges and Districts. The Department of Finance and the 
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California Postsucondary Education Commission were consulted during the 
development of the cost study and the draft plan for a statewide program. 



Technical Assistance Guide 

The technical assistance guide. Improving It: Accountability by Design, offers 
suggestions for Community College educators who wish to design accountability sys- 
tems to increase institutional proQvicUvity and effectiveness. Users are encouraged 
to view accountability as an jpportuaity to prove to themselves and others what their 
institution is accomplishing and tJ design ways to monitor and ensure continued 
success. The guide does net preFvribe the requirements for accounUbility; rather, it 
raises issues about the design of accountability systems which Community College 
staff will need to decide for themselves. Appendix B Contains a summary of the 
guide's recommendations. 



State and Local Costs 

The cost study, California Communtty College Accountability: State and Local 
Implementation Costs, was designed to provide recommendations fur the Chancellor's 
Office 1992-93 Budget Change Proposal (BCP) requesting additional funds for the 
implementation of a statewide accountability program. Earlier BCPs had requested 
substantial new funds for the stert-up and operation of local accountability programs. 
The findings of the cost study suggested that a study of implementation cosU is 
premature at this time. The reasons cited for a delay include: 

1. Considerable differences exist in utilization of Management Information 
Systems; 

2. Community Colleges have vastly different levels of institutional research 
capability; 

3. There is considerable variation in progress toward implementing a 
comprehensive accountability system; and 

4. Many districts are currently ill-prepared to engage effectively in a 
statewide accountability system. 

Although many Colleges are not now ready to implement accountability, there are 
others which, with the appropriate assistance, are willing and able to implement 
accountability programs. 

The cost study recommended that the focus of the 1992-93 BCP shift to building state 
and local institutional capacity for accountability, and that the Chancellor s account- 
ability program include the following five components with appropriate sUte level 
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budgetary support: refining basic data collection and reporting mechanisms, dcvel 
oping in-depth accountability studies, developing statewide surveys of student 
satisfaction and long term success, eliminating unnecessary or redundant data coUcc 
tion, and organizing a technical assistance information depository/ clearinghouse. 



Draft Statewide Accountability Program 

The objective of the draft statewide accountability program is to discuss state level 
accountability needs. Local needs are discussed in the design g^iide, Improving It: 
Accountability by Design. Although any state-produced accountability report will be 
shared with Colleges, it is not a substitute for local programs which fall under the 
authority of local boards. Of course, a strategy to coordinate state and local programs 
is essential. This strategy will require Chancellor's Office leadership for an inte- 
grated program of state and local reporting, and technical assistance to districts. 

The selection of performance indicators is an essential part of an accountability 
program. An indicator is defined as a measurable characteristic of the condition of 
the Conununity Colleges. There are many indicators that would be nice to collect and 
report, but restrictions on the number of indicators are dictated by the burden 
associated with the processing and use of information. Indicators should be selected 
to best represent the characteristics of Colleges that are related to desired student 
outcomes as outlined in the Basic Agenda and AB 1725. Appendix C contains a list o.* 
the indicators proposed for statewide reporting. Colleges are encouraged to augment 
the list of indicators proposed for statewide reporting in order to provide more 
meaningful and useful descriptions of local conditions. 



Staff l*roHcntation Jud\ Walh'rs. Vtcv Chancellor 

Hnlny Analysis 

Murk K. t'eder, Program Director 
I'lannirifi and Perfurmanve Outcomes U nit 
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Educational Accountability for 
California Community Colleges 

L Background 

Assembly Bill 1725 (Chapter 973, Statutes of 1988) mandates that . . the 
Board of Governors shall develop and implement a comprehensive Community 
College educational and fiscal accountability system . . . [Education Code 
Section 71020.51 in order to improve the quality of student and institutional 
performance. The system is to be implemented over a three-year period, 
beginning not later than 1991-92, and is to produce a published report of 
California Community College accountability. 

A task force report, AB 1725 Model Accountability System, was adopted by the 
Board of Governors in July 1990. The report described roles for Colleges, 
Districts, and the Chancellor's Office. The five recommended areas of 
Community College performance to be addressed include student access, 
student success, student satisfaction, staff composition, and fiscal condition. 
The purposes of a state level report include systemwide planning, policy, and 
budget formulation. Although primary audiences for a state-level report are 
the Legislature and Governor, the information will be available locally. 
Colleges will report to the Board of Governors and the Chancellor via existing 
information sources. Local accountability programs should relate to local 
planning, policy, and budget needs. 

An eighteen-month long accountability pilot program began in January 1991 
with $375,000 in grants awarded to four Districts (San Joaquin Delta, San Jose- 
Evergreen, Mt. San Antonio, and Santa Barbara) in order to produce prototype 
accountability reporU and strategies for improving the quality of student 
outeomes. A task force has been charged to provide consultation in the 
development of prototype accountability reports for Colleges, a guide for 
implementing local accountability programs, and a systemwide accountability 
program. (See Appendix A for a list of current task force members.) 

11. Local Accountability Program Design Guide 

The guide, developed under contract with Far West Laboratory, Improving It: 
Accountability by Desinn, was published in June 1991. It offers suggestions for 
Comm\«nity College educators who wish to design accountability systems to 
increawe institutional productivity and effectiveness. Users are encouraged to 
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view accountability as an opportunity to prove to themselves and others what 
their institution is accomplishing and to design ways to monitor and ensure 
continued success. The guide does not prescribe the requirements lor 
accountability; rather, it raises issues about the design of accountability 
systems which Community College staff will need to decide for themselves. 

The guide stipulates that a basic abpect of designing an accountability system is 
recognizing that the system should serve the information needs of both the state 
and the institution. The state needs information to assess whether the 
systemwide mission of the Community Colleges is being achieved. The 
institution needs information to determine where and how to improve teaching, 
learning, and support services. 

An accountability system provides information about the extent to which 
College policies and practie;s are achieving their intended objectives. The 
information from an accountability system should assist College staff in 
strengthening the institution. The indicators should be based on agreed 
standards of achievement, should be specific and focused to enable College 
personnel to better manage the institution, and should be integrated into the 
existing decision-making mechanisms of the College. 

One of the most politically sensitive decisions that the state and the Colleges 
can make is how to make use of accountability data. How will Colleges be 
compared? How will accountability data affect the allocation of funds? 
Ultimately, deciding how statewide accountability data will be used by 
policymakers needs to be agreed to by the Colleges, the Chancellor's Office, and 
various state agencies. 

Not all of the Community Colleges are equally ready to implement an advanced 
accountability system. They may differ considerably in vision or staff 
commitment to accountability, information-gathering capability, available 
resources, technology, or coordination between institutional review and 
planning processes. 

An accountability system can be a powerful tool. It can provide data, 
particularly outcome measures, which influence the thinking of faculty, 
administrators, and the community. Like any powerful tool, an accountability 
system is only as good as its design and implementation. The most effective 
systems are periodically revised to reflect changes in technology and the 
political context. By continually working to link accountability systems to 
institutional assessment, planning, and improv nent. the Community Colleges 
will ensure assessment and planning that reflect the richness and diversity of 
their institutions. 

The guide makes a number of rcctHnmendatit)ns regarding acctuinLahiliLy 
design principles, key decisions, criteria fur selecting indicaWrs, the need for a 
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vision of accDunUbility, the uses of information, lucbnology and infrastructure 
needs, and linkages with planning. These recommendations are summarized m 
Appendix B. 



III. State and Local Implementation Costs 

In March 1991, the Chancellor's Office contracted with Strategic Planning 
Associates (SPA) to undertake a study of AB 1725 implementaUon efTorls in the 
four Community Colleges designated as accountability pilots. A major purpose 
was to develop appropriate cost information which could serve as the rationale 
for Budget Change Proposals (BCPs) the Chancellor's Office might wish to 
pursue regarding both state and local accountability implementation. The 
initial intent was to focus on costs and funding required for successful 
implementation. The report ended up focusing as well on a more critical issue, 
the system's readiness for implementation. 

To examine these issues, the study relied upon the following analytic 
procedures: 1) review of documents provided by the Chancellors Office, 
2) interviews with key state and local officials, 3) attendance at two full-day 
meetings with the Chancellors Accountability Pilot Task^Force, and 4) site 
visits to 14 Community Colleges, including each of the pilot Colleges. 

The decision to visit fourteen campuses represented a modification of the 
workplan. Originally, it was hoped that site visits to the four pilot projects 
would be sufficient to generate the cost requirements associated with AB 1725 
accountability. This turned out not to be possible since Districts had only begun 
their projecU and had not collected appropriate cost data. The study was 
altered to include a broader array of Colleges and shifted focus to examining 
College "readiness" for accountability. 

Four principal findings formed the basis for subsequent recommendations: 

1. considerable differences exist in utilization of Management 
Information Systems; ^ . . . , 

2. Community Colleges have vastly difi*erent levels of institutional 

research capability; 

3. there is considerable variation in progress toward implementing a 
comprehensive accountability system; and 

4. many Districts are currently ill-prepared to engage effectively m a 
statewide accountability system. 

Although many Colleges were not ready for the adoption of accountability 
measures, there are many others which, with the appropriate assistance, are 
willing and could be able to adopt effective accountability systems. 
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The next phase of the investigation identified characteristics of Colleges that 
had successfully implemented accountability procedures. Each of these 
institutions had a: 1} strong vision of the role of accountability, 2) relatively 
advanced technical ability, and 3) commitment to provide sufficient resources. 

The study concluded, despite the utility of some of the cost information, that 
attempts to build sophisticated cost models are premature. It is recommended 
that the focus shift instead to building state and local institutional capacity for 
accountability, and that the Chancellor's accountability system include the 
following five components with appropriate state-level budgetary support: 

1. Kasic Data Collection and Keporting. The Chancellor's Office 
would annually compile information contained in MIS tapes and 
other statewide data rouUnely collected by the Chancellor's Office. 
The items, to be compiled and formatted for this portion of the report 
to the Legislature, would be determined by the Chancellor, after 
consultation with the Chancellor's Statewide Accountability Pilot 
Task Force. 

2. In-depth Accountability Studies. In addition to the basic data 
report, the Chancellor's OfTce would prepare a long-range plan to 
cover in-depth, each year one of the five accountability areas 
stipulated in the Hoard of Governors accountability program: 
student access, student success, student satisfaction, faculty 
diversity, and fiscal condition. 

3. Statewide Surveys. The Chancellor's Ofilce periodically would 
conduct a statewide survey to assess long-range program effective- 
ness and student satisfaction. 

4. More Kffectlve Data Collection and Dissemination Kfforts. The 
Chancellor's Office would develop a long range plan to reduce the 
response burden placed on Districts by multiple information 
demands. 

5. Depository/Clearinghouse. The Chancellor's Office would also 
serve as a depository for exemplary accountability models and 
provide technical assistance to local Districts as they embark on 
establishing their own local accountability models. 

To fund this activity, the cost study recommended that the Chancellor's Ofiice 
budget be augmented by approximately $550,000. This relatively modest 
investment in improving the capacity at both the state and local level to 
respond to the demands for accountability repr^-sents an infinitesimal 
percentage of the over $2.5 billion currently invested by California's citizens in 
Community Colleges. 



Commitment to Quality 5 



IV. Proposed Chancellor's Accountability Program 
A. Introduction 

The proposed state-level program is intended to serve primarily the needs 
of state policymakers and the public. This intent suggests several design 
requirements. Any reported information must be: quantifiable, 
intelligible to the lay public, focused on core features, and sufficiently 
stable to document trends. Commitment to improved quality requires a 
system that goes be.vond the mere collection and reporting of data. If the 
benefits of accountability are to justify the costs and the system is to work, 
there must be a mechanism to use this information in order for the State 
and Colleges to set improvement goals, form plans, guide implementation, 
and to understand outcomes. Discussed below are some of the reasons why 
existing management information system (MIS), accreditation, program 
review, and compliance activities will not fully meet the needs of this 
state- level accountability program. 

An efficient, high quality, MIS is only one, albeit very important, part of 
an accountability program. Major accountability activities that typically 
are not addressed by the MIS include the systematic use of information to: 
set quality improvement goals, plan strategically, and implement policies 
effectively. 

Voluntary, nongovernmental, institutional accreditation is widely 
recognized as a means of establishing minimum standards of quality for 
Colleges and of fostering educational excellence. Currently, accreditation 
relies primarily on qualitative data, works with a sample of Colleges each 
ear, and operates on a five-year cycle. Accreditation visits are oriented to 
the needs of individual Colleges and not to the system as a whole. An 
effective statewide accountability program requires that qualitative 
results be supported with objective quantifiable results, that all 
institutions be examined at the same time, and that the improvement 
needs of the entire system be addressed. 

At the Suite level, compliance and program review activities tend to be 
framed in terms of minimum standards for funding and levels of service 
for particular programs. The purpose of compliance and program review is 
to check whether standards are met, with an eye to corrective action. 
While minimum standards help assure provision of recommended levels of 
service, an effective accountability program must look beyond minimum 
standards to define areas for ongoing improvement and must encompass 
the entire institution, not just particular progxams. 

Despite these limitations, accountability activities must be coordinated 
with MIS, accreditation, compliance, and program review activities in 
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order to build upon existing structure and to avoid conflicting or 
redundant requirements. One goal of accountability is tbe efficient 
collection of information by the Chancellor's Office, However, state level 
indicators do not replace accreditation, research, program evaluation, or 
compliance programs. By analogy, a general practitioner M.D. begins to 
diagnose a patient's condition by observing general indicators of health. 
More detailed testing may be done by specialists. 



K. A Draft State Program 

The five proposed components of a State accountability program are 
described by Strategic t'ianning Associates in their report of state and 
local implementation costs. 

"What is needed is a new paradigm for California Community 
Colleges Accountability. Such a reconfiguration would respond 
to the often competing state and local interesU and would 
emphasize state leadership and technical assistance in 
achieving a meaningful, efi'ective, and useful accountability 
1 /stem 

Basic Data Collection and Reporting 

The Chancellor's Office would annually compile information 
contained in MIS tapes and other statewide data routinely 
collected by the Chancellor's Office. The items, to be compiled 
and formatted for this portion of the report to the Legislature, 
would be determined by u^e Chancellor, after consultation with 
the Chancellor's Statewide Accountability Pilot Task Force, 
and would be consistent with the provisions of Assembly Bill 
1725, the Board of Governors Basic Agenda, and the AB 1725 
Model Accountability System. The objective is to be responsive 
to the statutory requirements without being overly 
burdensome on Districts. As new phases are added to the MIS, 
this portion of the reporting requirement can be enhanced, but 
to the extent possible, these data elements should remain 
stable so that judgments about system progress over extended 
periods of time can be made. For this portion of the report, 
Colleges would not be required to submit any informatjon not 
already collected. All of the compiling, editing, and formatting 
would be done from existing data tapes in the Chancellor's 
Office. SimuiUneously, with the release of the information to 
the legislature. Community College district officials would 
receive a copy of the statewide report, along with a 
disaggregated report on the performance of their College(s) on 
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thesalected measures. Thes© nft*icdato eiemwilsoDulff then be 
au|^n»cnted by local inlonnation lo form the bftsis for Imal 
accountability reports tailored to each community. 

This basic report would be directly responsive to the needs of 
state officials to retrieve usable information from the MIS for 
which the state provided at least partial funding. At the same 
time, by limiting data elements to those required by MIS, this 
aspect of the plan is responsive to local concerns about data 
burden. No information, not already collected, would be 
required. 



In-depth Accountability Studiea 

In addition to the basic data report, the Chancellor's Office 
would also prepare a long-range plan to cover in-depth each 
year one of the five accountability areas stipulated in the Board 
of Governor's acc /untability program: student access, student 
success, student satisfaction, faculty diversity, and fiscal 
condition. These in-depth studies would be contractci iy the 
Chancellor's office through an RFP process and would involve 
topics to be determined by the Chancellor, after consuluUon 
with the Chancellor's Accountability Pilot Task Force, the 
California Postsecondary Education Commission, the 
Department of Finance, and the Office of the Legislative 
Analyst. Research would be focused on topics designed to 
produce information which would lead to program 
improvement. 

As an example, in addition to the annual report on the number 
of high school graduates who enroll in Community Colleges 
directly out of high school, research could be conducted to 
identify exemplary Community College programs which had 
proven success on this dimension. As another example, while 
the basic data will contain information about the numbers of 
students currently being served in various categorical aid 
programs, in-depth studies would be needed to ascertain the 
numbers of students needing services, but not currently being 
served. In the area of faculty diversity, an appropriate research 
topic could be to analyze the current status of faculty diversity 
and to identify and disseminate exemplary practices of 
Districts which operate successful programs in enhancing 
faculty diversity. 
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Statewide Surveya 

The Chancellor's OfTicj would periodically conduct statewide 
surveys to assess student satisfaction and long-range program 
effectiveness. These surveys could be designed to assess 
current Community College students as well as the general 
public. Student surveys similar to those conducted for the 
Chancellor's Office by Mervyn Field in the early 1980s could 
generate important state-level information that would be 
highly useful for both state and local policy makers. Since most 
Califurnians have attended a California Community College, 
general public opinion surveys could be designed to effectively 
yield longitudinal information on the effectiveness of 
Community College programs and public perceptions of 
Community Colleges. The Chancellor's Office would contract 
through an RFP process for surveys to be conducted on a 
regular (perhaps every three years) basis. These surveys would 
provide meaningful statewide information about student 
satisfaction (this information could be drawn from the periodic 
student surveys), longitudinal impact of the Community 
College program on student outcomes (this information could 
be generated from public opinion surveys). At the same time, 
test designers would be required to provide local Conununity 
College Districts with useful item banks which could then be 
used for similar kinds of local surveys. These surveys are often 
prohibitively expensive for local Districts to undertake, but for 
the state's overall accountability purposes and utilizing 
sophisticated sampling techniques, they can yield very useful 
information for relatively little cost. 

There are two additional purposes served by such a strategy. 
First, by making item banks available to local Districts, the 
state can provide useful assistance in the improvement of local 
student satisfaction and public opinion surveys. Second, by 
funding a statewide model, the state can encourage the 
development of local instruments which would be more closely 
linked to the state surveys, and hence insure greater 
comparability. 

More Effective Data Collection and Dissemination Efforts 

One of the most frequently cited complaints in site interviews 
focused on the issue of irrelevant or redundant information 
requests. In response to these concerns. SPA proposes that the 
Chancellor's Office develop a long range plan to reduce the 
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response burden placed on Districts. The long term goal v i 
to eliminate ali duplicative and redundant informa^»un 
requests. The Chancellor's Office would rely on the work ot an 
advisory committee composed of both state and local olUcials as 
well as the Chancellor's AccounUbility Pilot T^-.k Force to 
review and recommend elements to be eliminated. The 
underlying concept is that whatever piece of information the 
district is required to report should only be required once and 
the Chancellor's Office should bear the responsibility of 
coordinating data reporting. The plan itself would be 
submitted to a periodic accountability test. Districts could be 
asked on a regular basis to evaluate the Chancellor's Office 
efforts to reduce the reporting burden. 

Sites also argued strongly for a system which linked Colleges 
and the Chancellor's Office electronically. If data could be 
transmitted electronically rather than the cumbersome tape 
process now being used and if a College could be able to perform 
edit and referential checking from the College itself, the system 
would be more efficient and the burden on Chancellor's Office 
staff for this function could be materially eased. 

Another problem frequently cited is that data are reported to 
Districts, but often not in readily usable formats. A commonly 
cited example of reports needing improvement are the annual 
segmental reports on the performance of Community College 
students who transfer to the University of California and the 
California State University. The Chancellor's Office could 
work with the California Postsecondary Education 
Commission, the University of California, and the California 
Slate University to receive and develop sysiemwide transfer 
information that could be disaggregated on a College-by- 
College basis designed to be more readily usable by local policy 
makers. The current format, designed for ease of reporting by 
the other segments is often underutilized because its guiding 
purpose is reporting and not program improvement. As 
mentioned above, a common core of student satisfaction items 
could be developed and maintained. 

Another example might be the development at the state level of 
a model methodology responsive to the new Perkins Act 
requirement for documented student follow-up. The current 
Perkins Act will require greater performance outcome data to 
evaluate the effectiveness of Vocational Education Act funds 
than at any time in the past. Other states utilize 
unemployment insurance information as one way of gaining 
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information about graduates of certain kinds of vocational 
programs. Although there are serious flaws with this 
technique, using it in coiijunction with other accountability 
measures could be a great improvement on the current 
accountability mechanisms which are either nonexistent or 
woefully inadequate. 



DepositorylClearinghouse 

Finall.v, the Chancellor's Office can serve as a depository for 
exem 'iary accountability models and provide technical 
assistance to Districts as they embark on establishing their 
own local accountability models. The Chancellor's Office now 
serves as an effective depository for College and district policies 
relating to program review. That office has provided a useful 
document summarizing various features of the implementation 
of program review. A similar kind of document on the subject 
of accountability would have immediate utility for Community 
College Districts and would augment nicely the implementa- 
tion guide Improving It: Accountability by Design prepared by 
Far West Laboratory, In addition, this office could provide a 
"broker" service by providing linkages between district 
personnel who need assistance with district personnel who 
have exemplary programs." 



C. Statewide Kducational Indicator Issues 

An indicator is a measurable characteristic of the condition of the 
Community Colleges. There are a limitless number jf indicators that 
would be interesting or nice to have. FiniU resources for data collection 
and analysis restrict the number of indicators that can be produced. 
Educators must balance the amount of time required to digest information 
with the need to make decisions and gef on with work. Indicators should 
be selected to best represent the characterstics of Colleges that are related 
to desired student outcomes. A draft set of indicators is displayed in 
Appendix C. 
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I). Summary 

The development of the California Community Colleges accountability 
program required by Assembly Bill 1725 is well underway. Accomplish- 
ments to date include: 

• The formation of a pilot program task force and the award of 
grants to Districts in order to develop accountability programs for 
the improvement of teaching and learning. 

• A technical assistance guide, improving It: Accountability by 
Design, for Colleges. 

• A cost study with a detailed five-point plan for developing state 
and local accountability programs. 

• A draft set of educational indicators for statewide reporting. 

The pilot program ends in June 1992. At that Ume, a final evaluation report 
will be prepared for the Board of Governors, which will contain recommen- 
daUons for further implementaUon of state and local accountability programs. 
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Accountability Pilot Task Force Members 



Peter Macllougali 
Chairperson 

Superintendent/President 
Santa Barbara 
Conununity College District 

Dorothy Bray 

Vice President 

Educational Services 

Desert Community College District 

Kobert Clarke 

Student Representative 

Council of Student Body Governments 

College of San Mateo 

Margaret Domioici 
Vice President 
Student Personnel Services 
Shasta-Tehama-Trinty Joint 
Community College District 



Clair Parsh 
President 

Community College Association 



Jose Peralez 
Assistant Director 
Human Resources 
Santa Monica College 

John Petersen 
Executive Director 
Accrediting Commission for 
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APPENDIX B 



Improving It: Accountability by Design 

Summary of Recommendations 

Key Accountability System Design Principles 

• Clarify the purpose of the system and develop consensus among all 
stakeholders. 

• Use accountability data for internal planning and decision-making. 

• Develop a long-range plan to design and implement the system. 

• Select indicators that are compatible with systemwide goals. 

• Use as many indicators as are practical. 

• Have an independent group monitor system performance. 

• Make accountability data understandable to various stakeholders. 

Key Design Decisions 

• Decide the type and number of indicators to be used. 

• Decide at what levels of the college the data will be used. 

• Decide how the data will be aggregated and reported for internal use and 
the state. 

• Decide how to report to various stakeholders. 

Criteria for Selecting indicators 

• Measure the core features of the institution. 

• Measure outcomes that are actually intended. 

• Provide information that is timely and relevant to policymakers. 
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• Focus on the college site. 

• Allow for fair comparisons. 

• Maximize the usefulness of data. 

Accountability System Implementation Issues 

• Develop active leadership to design the system, from top administrators to 
technical personnel. 

• Decide the focus of the system, i.e., management, faculty, or students, and 
involve constituents in its planning and operation. 

• Clarify who has access to the data and how access is provided. 

• Clarify the kinds of daU needed for ongoing improvement. 

• Clarify how data will be shared with participants in ways that respect 
ethical considerations and confidentiality. 

• Develop qualitative measures of the dynamic aspects of college life; identify 
the areas where qualitative data are most needed. 

• Design a "value-added" system to generate needed information and benefit 
participants, e.g., desktop access. 

• Develop data collection processes to ensure accurate information for 
statewide reporting, e.g., student transfer. 

• Develop statewide consensus about how data will be used, i.e., types of 
comparisons and access. 

• Clarify the relationship between tiie college's system and the Chancellor's 
MIS; facilitate local access to statewide MIS daU. 

• Standardize computer hardware and software to facilitate transmitting and 
sharing data within the college. 

• Improve access to data stored in the college's computers. 

• Allow adequate resources to transform data into forms easily read and 
understood by nontechnicil people. 

1.') 
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Communicate how the demands generated by the accountability process will 
add to or replace requirements already in place. 

Communicate how the accountability system will benefit participants and 
help the institution function better. 

Communicate how accountability will affect resource allocation, sanctions, 
and the development of policies and practices. 
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Draft Statewide Indicators 



The indicators proposed below are primarily of statewide interest. Colleges, at their 
discretion, may wish to augment them in order to provide more meaningful and 
useful descriptions of local conditions. The indicators are arranged in the five 
categories prescribed by the model already submitted to the Board of Governors: 
student access, student success, student satisfaction, faculty diversity, and fiscal 
condition. 

Measures of student outcomes are difficult to interpret without appropriate back- 
ground infv^rmation. Student outcomes are affected by context, inputs, and processes 
as described below. Definitions note the role of each indicator as context, inpnt, 
process, or outcome and also note which of the measures proposed by AB 1725 (A 
through K) are addressed. 

• Context relates to the environment in which Community Colleges operate. 
These factors are not easily manipulated. For example, demography, busi- 
ness needs, or conmiunity needs. 

• Input refers primarily to the fiscal resources available to Colleges. Decision 
makers have some control over inputs. For example, funding, and federal, 
state, or local policies. 

• Processes include the plans and activities intended to meet student needs. 
For example, curriculum, instructional services, student services, adminis- 
tration, management, training, or communications. 

• Outcomes encompass student goals. For example, jobs, transfer, degrees, 
certificates, or student satisfaction. 



Proposed Indicators 
1. Student Access 

1 .1 General Participation 

Definition : Number of students enrolling. Context measure. AB 1725A, 
B. 

Format: Display by gender, ethnic category, high school graduation 
status (recent graduate or not), age group, and fall versus spring. 
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Justification : The number of students enrolling in the Community 
Colleges is a basic measure of access. It also has implications for the 
amount of resources needed 1 provide adequate levels of service. 

.» 

Availability : MIS 

1 .2 Transition from High School 

Definition : Percent of high school graduates enrolling within two years 
following graduation. Context measure. AB 1725 A, B. 

Format : Display by term, gender, and ethnic category. 

Justification : High schools are one important source of Community 
College students. The flow of students from high schools to Community 
Colleges is useful information for the planners of articulation programs. 
Most high school graduates who enroll in a Community College do so 
immediately after graduation. The percent of recent high school 
graduates who enroll in the fall immediately following graduation is a 
leading indicator of the eventual total number enrolling. Free flow may 
affect the usefulness of this information locally, but not statewide. 

Availability : MIS. Statewide graduation counts must be obUined from 
California Department of Education. 

1.3 Financial Aid 

Definition : Number of recipients and aid per recipient in constant dollars. 
Input/process measure. AB 1725 A, K. 

Format : Display by source (federal, state, local) and type (grant, loan). 

Justification : Low income and disadvantaged students frequently require 
financial assistance in order to attend College. It is believed that not all 
students who actually need financial aid also apply for it. One problem is 
the lack of a commonly accepted, objective definition of "need." The 
number of fee waivers granted would be useful background information. 

Availability : MIS. More detailed information on student need could be 
obtained through a statewide survey. 

1.4 Categorical Programs 

Definition : Number of students served and average expenditure per 
student. Process measure. AB 1725 A, I. 
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Format : nj^play by sex, ethnic group, and program, including EOPS, 
DSP&S, Matriculation, and CARE. 

Justification : Categorical programs are designed to assist economically or 
academically disadvantaged students who face obstacles in effectively 
accessing education. A problem is the lack of an objectively quantifiable 
definition of need for these services and a measure of the amount of unmet 
need. 

Availability : NCS. This information is scheduled for a later phase of the 
MIS. It should be possible to obtain partial results from existing reports 
supplied to the Chancellor's Office. More detailed information could be 
obtained through the longitudinal survey. 

1 .5 "^asic Skills and ESL Enrollment 

Definition : Number of students enrolled each year in basic skills and ESL 
courses. Context/process measure. AB1725H. 

Format : Display by sex, ethnic group, and course. 

Justification : Student success in academic or vocational education 
depends on the mastery of certain basic skills and of English. The overall 
level of student skills at a campus also affects the quality and nature of 
instruction that can be provided. 

Adequacy of basic skills and ESL programs can be framed in terras of 
student need and subsequent student success. While the determination of 
adequacy is important, it is also quite subjective. 

A limitation of this indicator is lack of information about the amount of 
remedial work needed prior to College level. Although difficult to collect, 
the average time in remedial required before entry into College-level 
English or mathematics would be of interest. 

Availability : MIS. Some data may be available from existing Chan- 
cellor's Office reports. 

Student Success 
2.1 Persistence 

Definition ; Number of students who are enrolled for two consecutive 
terms. Outcome measure. AB1725D. 
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Format ; Display by sex, ethnic category, full versus part-time, and high 
school graduation status. 

Justification : Persistence in meeting specified requirements is a 
necessary jonc'.ition » " obtaining a certificate, degree, or transfer. 

One cannot reasonably evpeci that all enrolled students should persist for 
a full year. Some students enroll in Conununity College programs or 
courses that require one year or less to satisfy their training or 
information needs. While not all student goals will require a full year, the 
completion of a degree, obtaining of some certificates, or satisfaction of 
transfer requirements will take a longer period of time. 

The definition of a suitable persistence rate would require the 
identification of a subgroup of students that can be reasonably expected to 
persist for a full year, e.g., those with the matriculation goal of a degree or 
transfer, who have satisfactorily completed English lA (Baccalaureate- 
required, first-term composition). Satisfactory completion of English lA 
can be determined via the proposed Subject Matter Taxonomy Codes 
which MIS anticipates will be adopted. TOP codes may not permit this 
level of discrimination. A new system of classifying programs, the 
"Classification of Instructional Programs" (CIP) code, sponsored federally, 
is under consideration at this time and may supplant the TOP codes. 

Availability : MIS. Numbers of persisting students should be obtainable 
from MIS. 

2.2 Completion 

Definition : Number of students each year who complete an associate 
degree, certificate, or vocational award within two, three, four, or more 
years of initial enrollment. Outcome measure. AB 1725 D, E, F. 

Format : Display sex, ethnic category, and high school graduation status. 
For certificates distinguish tv^u-year, one-year, and less than 1 year. 

Justification : Attainment of a degree or certificate is an important result 
of Community College education, depending both on access and 
persistence of students. 

Student transfer is a type of completion that is? not measured by this 
particular indicator. Some students who transfer do not complete their 
degrees. 
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Availability : MS has the number of awarded degrees. The numbers of 
transfers should bs available from CPEC, or directly from UC and CSU. 

Transfer 

Definition : Number of students who transfer to the University of 
California or the California State University within two, three, four, or 
more years. Outcome measure. AB 1725 C, D. 

Format : Display by sex, ethnic category, and years needed. 

Justification : Community Colleges provide students with the opportunity 
to transfer to a four-year college. 

Consensus on the definition of a transfer rate has proven to be elusive, not 
only for reasons of data qua Jty but also because of disagreement over 
basic concepts. The available transfer data are subject to a number of 
criticisms. Much of the available data has been collected in accordance 
with institution specific, or segment specific, definitions which are not 
generalizable statewide. Most data collection systems only count 
transfers to public four-year institutions and are likely to undercount the 
actual number of transfers. Finally, the usual statistical analyses are 
frequently based on single-year data cross-sections which do not track 
individual students longitudinally over years. These data problems not 
only attack the validity of many computed transfer rates, they also 
undermine the capacity to investigate causes of trends and factors 
influencing transfer. 

A conceptual barrier to achieving consensus on a transfer rate definition is 
the disagreement on defining the pool of tran'^fer candidates which can 
serve as a denominator for the rate. A number of possible transfer 
candidawe pools have been proposed. Another issue is that the size of the 
numerator, the number of actual transfers, is limited by the number of 
slots available at four-year public institutions. Some of the more 
prominent alternatives are ketched below. 

a. Betman and Vi/eiler. The denominator for the transfer rate is the 
exiting cohort of students in a term - that is, those students who 
were enrolled for credit in one term and but did not re-enroll in the 
subsequent term. These students are called leavers. The transfer 
rate is the number of transfers divided by the number of leavers. 

b. Traditional Approach. One commonly used denominator is the total 
credit enrollment. The rate is the the number of transfers in a term 
divided by the total credit enrollment, regardless of whether all 
enrolled students are in fact able to transfer in that semester. A 



refinement is to look at the total credit enrollment some time prior, 
e.g., two years, to the date of transfer. 



c. CohenlCenter for the Study of Community Colleges. Credit 
attainment is examined in order to understand the student's intent 
to reach his/her educational goal. Two percentages are computed. 
One is the percentage of students from a first-time entering cohort 
who earned 12 or more credits during a four-year period (C/E). The 
other is the percentage of students obtaining 12 or more credits who 
transfer to a senior institution during a four-year period (T/C). 
Together both percentages represent the outcomes of the transfer 
process taking place within the institution. 

d. ICC Transfer and Articulation Committee. The transfer candidate 
pool consists of those students who complete 12 or more UC/CSU 
transferable credits in their first academic year. The percent of 
transfers from this cohort is computed every year thereafter until the 
annual number of transfers becomes negligible. A variation of this 
approach is to more narrowly define the transfer candidate pool to 
include only those students with a sUted intent to transfer in 

' ^ addition to earning at least twelve credits in their first year. (Please 
note that the ICC proposal is subject to modification at this time.) 

e. Other. Various other criteria for identifying student intent to 
transfer have been proposed, including completion of specific 
courses, for example English lA, or completion of all credit require- 
menU for transfer. Various transfer candidate pools can be defined 
in terms of those students who meet these criteria. 

Transfer is a very complex phenomenon. Successful transfer depends on a 
number of factors, including services provided by Community Colleges, 
student commitment, services provided by four-year institutions, and 
agreed upon intersegmenUl policies. This complexity suggests that 
multiple indicators be used to track transfer performance. Assuming that 
data quality issues can be resolved, it will be necessary to evaluate 
empirically the reliability of the transfer indicators and to verify that they 
in fact provide useful information about the pen jrmance they purport to 
describe. 

Availability : MIS, statewide survey, or Herman and Weiler database. 
Job Placement 

Definition : Number of students placed in jobs within six months of degree 
or certificate. Outcome measure. AB1725G. 
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Format : Display by gender, ethnic category, and nuijor field. Include 
student self-assessment of relevance of training. 

Justification : Community Colleges provide occupational preparation for 
students. 

The assessment of relevance to training can be highly subjective. Obtain- 
ing some jobs may not require a degree or certificate, but only completion 
of particular courses. 

Availability : Employment Development Department database. The 
Statewide Student Follow-up System (SSFS), funded by the Vocational 
and Occupational Programs Division of the Chancellor's Ofilce may be a 
potential source of this information. Currently, the SSFS is implemented 
on a voluntary pilot basis at a number of Colleges. 



3. Student Satisfaction 

3.1 -3.5 Student Expectations and Satisfaction With Community College 
Experience 

Definition : Expectations and satisfaction before, during and after 
attendance with 

(1) access, 

(2) instruction, 

(3) instructional services, 

(4) student services, and 

(5) facilities. 

Outcome measure. AB 1725 E, I. 

Format : Display by gender and ethnic category. 

Justification : Studente are the primary clients of the Community Colleges 
and their satisfaction is an essential measure of success. Quality can be 
assessed by comparing expectations with subsequent satisfaction. 

Measurement of satisfaction is a complicated matter and will depend on 
an appropriate survey. The benefits of such information must be weighed 
against the costs of gathering it. A carefully designed self-report ques- 
tionnaire as one part of a student survey is recommended as an economical 
approach. 
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Availability : Statewide survey. 

Staff Composition 

4.1 Staff Diversity 

Definition : Number and percentage of staff by gender, ethnic, and age 
group, the date of hire, Equal Emloyment Opportunity job category. 
Process measure. AB 1725J. 

Format : Display by age, gender, ethnic category, and Equal Employment 
Opportunity category. Compare to available workforce statistics. 

Justification : Staff" composition afi'ects campus climate, particularly for 
underrepresented students. This indicator is supportive of the equal 
employment opportunity policies and affirmative action programs that 
firmly are ingrained in the federal and state approaches to education. 

Availability : MIS 

4.2 Faculty Time Assignment 

Definition : Number and percentage of faculty working full-time or part- 
time. Process measure. AB 1725 J. 

Format : Display by gender and ethnic category. 

Justification : The use of full-time faculty provides for a more stable 
instructional climate and increased professionalism. This measure also 
supports Equal Employment Opportunity policies. 

Full-time faculty are more likely to keep regular office hours and serve on 
committees. 

Availability : MIS. 
F iscal Condition 

5.1 Community College Funding 

Definition : Overall level of funding in constant dollars for state and local 
operations. Input variable. AB 1725 K. 
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Format : Display funding adjusted for inflation by source (state, federal, 
local) both as a total and by ADA/FTES. Input measure. 

Justification : Funding is a core indicator of the support and commitment 
of government to the Community Colleges. 

Availability : MIS, Fiscal Services Unit, or Legislative analyst. 
5.2 Fiscal Stability 

Definition : Number of Districts rated as at fiscal risk. Input variable. 
AB 1725 K. 

Format : Display by fiscal quarter the number of Districts at high, 
medium, or low fiscal risk. 

Justification : Districts have a fiduciary trust in the handling of public 
money. 

Availability : Existing Chancellor's Office Fiscal Services Unit surveys of 
district fiscal status. 
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